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1. Introduction 

A celebrated theorem of Selberg [32] states that for congruence sub- 
groups of SL 2 (Z) there are no exceptional eigenvalues below 3/16. We 
prove a generalization of Selberg's theorem for infinite index "con- 
gruence" subgroups of SL 2 (Z). Consequently we obtain sharp upper 
bounds in the affine linear sieve, where in contrast to [I] we use an 
archimedean norm to order the elements. 

Let A be a finitely generated non-elementary subgroup of SL 2 (Z); 
let X\ = A\H be the corresponding hyperbolic surface (which is of 
infinite volume if A is of infinite index in SL(2,Z)). Let 5(A) denote 
the Hausdorff dimension of the limit set of A. The generalization of 
Selberg's theorem splits into two cases: 5(A) > | and < 5(A) < |. 

In the case that 5(A) > | the spectrum of the Laplace-Beltrami 
operator on L 2 (X\) consists of finite number of points in [0, \) (see 
[TT]). We denote them by 

< A (A) < Ai(A) < - - < A max (A) < -. 

The assumption that 5(A) > | is equivalent to A (A) < 4, and in this 
case 5(1- 5) = A [21]. 

The following extension of Selberg's theorem is proved in section [2j 

Theorem 1.1. Let A be a finitely generated subgroup o/SL(2,Z) with 
5(A) > |. For q > 1 let A(q) be the "congruence" subgroup {x E A : 
x = I mod q}. There is e = s(A) > such that 

X 1 (A(q))>X (A(q))+e, 

for all square-free q > 1 (note that Ao(A(g)) = \ (A)). 
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In [TT] an explicit and stronger version of Theorem 11.11 is proven 
under the assumption that 5(A) > |. See [29J for the sharpest known 
bounds towards Selberg's \ Conjecture as well as bounds towards the 
Ramanujan Conjectures for more general groups. 

Theorem 1 1.1 1 is a consequence of Theorem 1.2 in [1] and the following 
result, which is of independent interest. 

Theorem 1.2. Let A = (S) be a finitely generated subgroup o/SL(2,R) 
with 5(A) > |. Let {Ni} be a family of finite index normal subgroups 
of A. Then the following are equivalent 

(1) The Cayley graphs Q(A/Ni, S) form a family of expanders. 

(2) There is e = e(A) > such that Ai(A/iV f ) > A (A/A^) + e. 

The argument in section [2] establishes that [TJ =>T~2~j the implication 
l2l=>mis proved using Fell's continuity of induction in section 7 of [TT] . 
Theorem 11.21 generalizes results of Brooks [5J and Burger [6], [7] who 
proved it in the case of co-compact A. 

Combining Theorem 11.11 with Lax-Phillips theory of asymptotic dis- 
tribution of lattice points [TT] we obtain the following result, which is 
the crucial ingredient in the execution of the affine linear sieve in the 
archimedean norm. 

Theorem 1.3. Let A be a finitely generated subgroup o/SL(2,Z) with 
5(A) > |. Assume that q is square free and (q,qo) = 1, where q$ is 
provided by the strong approximation theorem [19]. There is £\ > 
depending on A such that for any g G SL^g) we have 

| {7 G A I ||7|| < T and'y = gmodq}\ 

(L1) = #r^T + ^ T2 ^ 1 )- 

|SL 2 (g)| 

We now turn to the discussion of the case 5(X) < |. In this case 
there is no discrete L 2 spectrum and its natural replacement is fur- 
nished by the resonances of X, which are given as the poles of the 
meromorphic continuation of the resolvent Rx(s) = (Ax — s(l — 
By the result of Patterson [25 and Sullivan [51] Rx(s) is analytic for 
3t(s) > 5; Mazzeo and Melrose [20J proved that Rx(s) has a meromor- 
phic continuation to the entire plane. In [25] Patterson proved that 
Rx(s) has a simple pole at s = 5 and no further poles on the line 
5R(s) = 5; his proof is based on ideas from ergodic theory related to 
Ruelle zeta-function. Using further development of these ideas due to 
Dolgopyat [8], Naud [2TJ has recently established that Rx(s) is holo- 
morphic (with the exception of simple pole at s — 5) for 3?(s) > 5 — e, 
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with e depending on X. The following result, giving a resonance-free 
region for congruence resolvent, is proved in section [TT1 

Theorem 1.4. Let A be a finitely generated subgroup o/SL(2,Z) with 
<KA) < \- For q > 1 square free let A(q) be the "congruence" subgroup 
{x G A : x = I mod q}; let X(q) = A(g)\H. There is e = e(A) > 
such that Rx( q ){s) is holomorphic( with the exception of simple pole at 

s = 5)for$l{s) >S-emm(l, ] ^ m y 

When 5 < | we cannot apply the expansion property [4] directly; 
instead, to prove theorem II .41 we use a dynamical treatment and invoke 
a generalization of the underlying result on measure convolution ( "L 2 - 
flattening lemma"): see Lemmas [1] and [2] in section [7J It is likely that 
by combining our methods with the extension of Dolgopyat's result [S] 
to vector- valued functions, analyticity of R X ( q )(s) can be established 
for 3?(s) > 5 — e — in complete analog;yQ in with Theorem 11.11 

Using methods of Lalley [16J we obtain the following analogue of 
Theorem 11.31 which is sufficient for sieving applications. 

Theorem 1.5. Let A be a finitely generated subgroup o/SL(2,Z) with 
< 5(A) < |. Assume that q is square free and (q,qo) = 1, where qo 
is provided by the strong approximation theorem [19]. There is E\ > 0, 
C > depending on A such that for any g G SL 2 (q) we have 

| {7 G A I ||7|| < T andj = gmodg}| 

(L2) -ra^ ^))^^")- 

We turn to applications to affine linear sieve [4]. Consider the stan- 
dard action on the two by two integer matrices by multiplication on the 
left, and take the orbit O of I (the identity matrix) under A. Set \x\ = 

(E M 4)" where * = Set ^a( T ) = I{^A: |x| < T}\ 

and let 6(A) be the Hausdorff dimension of the limit set of an orbit 



1 The analogy between Theorem 11.11 and Theorem 1 1 . 41 becomes clearer when their 
assertions are expressed in terms of the Selberg zeta function [31] . If A is a finitely 
generated subgroups of SL 2 (K) the Selberg zeta function Zx(s) associated with 
X = A\H is known to be an entire function, whose non-trivial zeros are given by 
the resonances and the finite point spectrum [T^l [5B] . Consequently, Theorem 11.11 
is equivalent to the assertion that when 6(A) > \ there is e(A) > such that 
Z X ( q )(s) is analytic and non-vanishing on the set {5?(s) > 6 — e}, except at s = <5 
which is a simple zero, while Theorem ll.4l is equivalent to the assertion that when 
6(A) < I there is e(A) > such that Z X ( q )(s) is analytic and non- vanishing on the 

set |?R(s) > 8 — emin ^1, j^iqqcjjjy^ |, except at s — 8 which is a simple zero. 
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Az C EI U {00} U R, where EI is the hyperbolic plane, z G H and A 
act by linear fractional transformations. By the results of Lax-Phillips 
P2] and Lalley [16] we have that N A (T) ~ c k T 2S ^\ as T -> 00. Let 
/ G Q[xjj] be integral on O and assume that it is weakly primitive 
for O, that is gcd{/(a:) : x G O} is 1. If / is not weakly primitive 
then j*f is, where N = gcd/(C), and we can represent any weakly 
primitive / as -hg with g G and N = gcd(C). 

The coordinate ring Q[xij] / (det(xij) — 1) is a unique factorization 
domain [28] and we can factor / into t = t(f) irreducibles /1/2 . . . ft in 
this ring. Set 

7T Ai/ (T) = \{x G A; \x\ < T,fj(x) is prime for j = 1, . . . ,t}\. 

For / G 1j[xij] weakly primitive with t(f) irreducible factors our 
conjectured asymptotics is of the form: 

(1-3) 7r A|/ (T) ~ (lQgr)f(/) , as T ^ 00, 

where c(A, /) can be expressed as a product of local densities; see [TUl 
[30] for an example of explicit computation and numerical experiments. 
In section [T3] we establish the following sharp upper bound for tt k j(T). 

Theorem 1.6. Let A be a subgroup o/SL(2, Z) which is Zariski dense 
in SL 2 and let f G Z[xy] be weakly primitive with t(f) irreducible 
factors. Then 

N A (T) 



;i-4) tt Aj/ (T) « 



(logT)*(/) 



We also obtain the following lower bound for the number of points 
x G A for which / has at most a fixed number of prime factors. 

Theorem 1.7. Let A be a subgroup o/SL(2,Z) which is Zariski dense 
in SL 2 and let f G Z[xy] be weakly primitive with t(f) irreducible 
factors. Then there is an r < 00, which can be given explicitly in terms 
of e(A) in theorems \1.1\ and\1.4\ such that 



(1.5) 

{s G A; \x\ < T, and f(x) has at most r prime factors}] 3> 



(logT)*(/)' 

2. Generalization of Selberg's 3/16 theorem when 5 > 1/2 

Being a subgroup of finite index in A(l), A(q) has the same bottom 
of the spectrum, A (A(g)\BT) = A (A(1)\H). As in section 2 of [H], 
we have that for q large enough A(l)/A(g) = SL 2 (Z/gZ). Let S = 
{Ai, . . . , A k }, and let S g be the natural projection of S modulo q. 
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Theorem 1.7 in |4] implies that if A = (S) is non-elementary, then 
Q q = <?(SL 2 (Z/gZ), Sg) is a family of expanders. Consider the space 
H(q) of vector- valued functions F on satisfying 



(2.1) 



F{ 1Z ) = R q {i)F{z), 



for 7 G A(l)/A(g) = SL 2 (Z/gZ), where Rqij) denotes the regular 
representation of SL 2 (Z/ qL) ; we denote by (, ) the inner product on this 
space and by || || the associated norm. Denoting by H (q) the subspace 
of functions in H (q) orthogonal to ip , the eigenfunction corresponding 
to Ao, the assertion of Theorem 11.11 is equivalent to existence of c > 
such that 



(2.2) 



U\\F\\ 2 d» 



> A + c 



for all F G H (q). 

Applying Theorem 1.7 in [3] for each z G J-"{1) implies that there is 
£ > 0, depending only on S, such that for all F G H (q), we have 



(2.3) 

Let / 
(2.4) 
where 

(2.5) 

and 
(2.6) 

Write 



\\F{jz) - F{z)\\ > e\\F(z)\\ for some 7 G S. 
\F\\, and decompose it as 

/ = a(p (z) + b(z), 



ip (z)b(z)d/j,(z) = 



\\1\ % dii 



a 2 + 



\b\ 2 dfi 



where N = |SL 2 (Z/gZ)|. Since 



F N {z)), 




(E^il^WI 2 )* 




if Ef=i \F,(z)\ V0 



otherwise, 



we have 



|VF|| 2 ( 2 )>|V||F||| 2 (^) = |V/| 2 (z). 
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Consequently we obtain: 
(2.7) 

UWVFfd^ j T \V\\F\\\ 2 dy _ j r \Vf\ 2 dy 
fr\\F\\*dn " b\\F\\*diA U\f\ 2 dfi " 



U\f\ 2 dv 



(a\ ip + Ab, a<p + b)dfi 



a 2 X + (Ab,b) > cfXv + ^J \b\ 2 dfi > A + (Ai - A ) J \b\ 2 d^. 

By a theorem of Lax and Phillips [17] there are only finitely many 
discrete eigenvalues of A in [0, j]; consequently 

(2.8) Ai - A > ci > 0. 

Therefore, as soon as J T \ b\ 2 dii > E\ > 0, we have that 

(2 - 9) tW^- Ao+ci£i - 

Now consider the case of \b\ 2 d[i = 0. We can assume that a = 1 
and write F(z) = u(z)(po(z), with u{z) = (ui(z), un(z)), where 
N = |SL 2 (Z/gZ)|. Now 

N 

(2.10) Hz)||=;>>,| 2 (z) = l 

implies 



and since 



dy Uj dy ^ <9y 



GENERALIZATION OF SELBERG'S THEOREM AND SIEVE 



7 



we have that 



HVwII 2 = 



(2.12) 



d<Po 
dy 



3=1 j=i 

2 N N 





\VF\\ 2 dfi 


Jjr 1 V(^o| 2 + (fl\\Vu\\ 2 dyL 


u 


\\F\\ 2 dn 


J T \fo\ 2 dfi 



3=1 

|Vv? | 2 + ^oI|Vm|| 2 . 

Consequently, 
(2 13) 

/jrkol 2 ^ XfI<A)| 2 ^ ~ Xrlv^ol 2 ^ 

Our aim now is to show that 

, x Lip 2 \\Vu\\ 2 dfi 
2.14) Jj t 01 , ' P > c 2 > 0. 

To that end we assume that that 

(2.15) k^f± <K 

and will obtain a contradiction for sufficiently small k (kj below are of 
the form aj ■ k for suitable constants aj). Consider the fundamental 
domain J-{1) = A(1)\D. Its boundary, <9jF(l), consists of finitely many 
geodesic arcs splitting into pairs lj, lj in such a way that there is 
7j G S so that lj = ^jlp 7j are distinct and generate A(l). Further, we 
have decomposition of the following form: 

JF(1) = /C(l) U [J cuspi U [J flare, 
iecu(i) jeF/(i) 

where 

(1) JC(1) is relatively compact in D 
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(2) Cu(l) is a set of cusps of ^(l). Each cusp^ is isometric to a 
standard cuspidal fundamental domain P(Yi) of the form 

P(Y) — {z — x + iy | < x < l,y > Y}, 

based on a horocycle 

h Y = {x + iy | y = Y}. 

(3) Fl(l) is a set of flares of F(l). Each flare^a) is isometric to a 
standard hyperbolic fundamental domain F(a) of the form 

F(a) = {z : 1 < \z\ < exp(/?); < arg(z) < a] , 

where a < ~. 

Since ip G L 2 (J r (l)), we have that 



(2.16) / \(fo\ dn> C3 / |v?o| dpi for some C3 > 0, 

Jk jt 

and therefore (12.151) implies that 

We recall the definition of Fermi coordinates. Let 77 be the geodesic 
in the hyperbolic plane parameterized with the unit speed in the form 

t -> r}(t) g e 2 i el. 

Then 77 separates H 2 into two half-planes: a left hand side and a right 
hand side of 77. For each p G H 2 we have the directed distance p from 
p to 77. There exists a unique /; such that the perpendicular from p to 
77 meets 77 at r/{t). Now (p , t) is a pair of Fermi coordinates of p with 
respect to rj. In these coordinates the metric tensor is 

(2.18) ds 2 = dp 2 + cosh 2 pdt 2 . 

Introduce Fermi coordinates based on the bounding geodesies lj, and 
use them to foliate /C. By compactness, using (12.171) . we can find z G /C 
and S > such that 

(2.19) / |<p | 2 > c 4 > 0, 

JB(z,8) 

and for all j — 1 , . . . , k 

L fx -, y?o II Vn II 2 dp, 

(2.20) JT f 1 1 " < ^ 

where Tj(5) is a tube lying in fC and containing B(z, 5) along the per- 
pendicular to lj. 
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Each T is of the form [—5,5] x [pij,P2,j] i n the appropriate Fermi 
coordinates. Rewriting (12.201) in Fermi coordinates (12. 181) . and using 
the fact that 

/ duj \ 2 ( duj \ 2 ( duj \ 2 
\dp~) + \dt) ~ \dp~) ' 
by Fubini's Theorem we obtain 

(2.21) / v l\\Vu\\ 2 d(i > 25 / ^ 2 \\u'{p)\\ 2 cosh(p)dp. 

Let L denote the maximal length of the tubes Tj. Using the fact 
that if \u{ Pl ) - u { p 2 )\ >Cand Pl -p 2 <L then \u'(p)\ 2 dp > C 2 / L 
(since 

J u'(p)dpj < l^j ldp) [J \u'(p)\ 2 dpj), 
we obtain that (I2.20j) implies that for all j — 1, . . . , k we have 

(2.22) / <Po\\u(jjz) - u(z)\\d/i(z) < « 3 / <pldp(z). 

JB(z,S) JB(z,8) 

On the other hand, since F{z) = u(z)ipo(z) and ipo^z) = <fo(z) for 
all 7 G SL 2 (Z/gZ), (12 .3p implies that there is e(S) > independent of 
q, such that 

(2.23) (^(t 2 ) — || > e(S) for some 7 G S. 

Applying mean-value theorem, we see that (I2.22p implies a contradic- 
tion with (12.231) once k is small enough depending on e(S); consequently 
we have proved the validity of ( 12. 14ft and the proof of Theorem 11.11 is 
complete. 

The adaption of the preceding argument to proving the implication 
[1] =>l~2l of theorem 11.21 is straightforward, as is the generalization of 
this result to higher dimensional hyperbolic spaces: the theorem of 
Lax and Phillips, of which we made crucial use in the first part of 
the argument, holds for geometrically finite subgroups of SO(n, 1) with 
Hausdorff dimension of the limit set greater than n/2; the second part 
of the argument proceeds as above by restricting to compact part of 
the fundamental domain and foliating it using Fermi coordinates. In 
particular, by combining the H 3 analogue of Theorem 11.21 with [37] 
and Theorem 6.3 in [1] we obtain the following theorem which has 
applications to integral Apollonian packings [101 [15| [30]. 
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Theorem 2.1. Let A be a geometrically- finite subgroup o/SL 2 (Z[\/— 1]) 
with 6(A) > 1 and such that the traces of elements of A generate the 
field Q(v / ^T)- There is e = e(A) > such that 

Ai(A(^))>A (A(^))+e 

as A varies over squarefree ideals in Z[a/— 1]. 

3. Counting lattice points for 5 > | 

Recall that the Poincare upper half-plane model is the following sub- 
set of the complex plane C: 

H 2 = {z = x + iy e C I y > 0} , 

with the hyperbolic metric 

1 

? 

The distance function on H 2 is explicitly given by 



(3.1) ds 2 = —(dx 2 + dy 2 ). 



\z 


— w\ + 


\z 


— w\ 


\z 


— w | — 


\z 


— w\ 



(3.2) p(z, w) = loj 

We will use the following expression: 

(3.3) cosh p(z,w) = 1 + 2u(z,w), 
where 

/ , . \z — w\ 2 

3-4 u(z,w)= l ——±. 

Akszksw 

The ring M 2 (M) of two by two real matrices is a vector space with 
inner product given by 

(g, h) = trace(g/i*). 
One easily checks that \\g\\ = (g,g)^ is norm in M 2 (M) and that 

(3.5) \\ g f = a 2 + b 2 + c 2 + d 2 iorg=(^ c \ 

By taking z = w = i in (13.41) we obtain that 

(3.6) ||#|| 2 = a 2 + b 2 + c 2 + d 2 = 4u(gi, i) + 2. 
Now the result of Lax and Phillips [T7j is as follows. Let 

(3.7) N A (T; z, w) = #{7 G A : p(z, jw) < T}. 
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Suppose 5 > \ and write Xj = 6j(l — 8j); 5 = 5. Denoting the 
eigenfunctions corresponding to Xj by (fj we have 

(3.8) \N{T- z,w) -Y,cm(z)vA^)e 5 > T \ = 0{T^e T ' 2 ). 

j 

Turning to congruence subgroups A(g) we have, using the methods 
of pi], that 

(3.9) \N A(q) (T;z,w) -^^ M ( Z )^Re^| = 0(g 3 T 5 / 6 e T / 2 ), 

j 

where implied constant is independent of q. 

The base eigenfunction for A(g), normalized to have L 2 norm one, is 
given by 

(3 ' 10) ^ = isl^zm)!^ 1 - 

Combining Theorem ll.il with (13.91) and (13.21) . (13.61) we obtain e\ > 
depending on A such that for any g G SL 2 (g) we have 

I {7 e A I || 7 || < Tand 7 = ^modg}| = + 0(g 3 T 25 " £l ), 

establishing Theorem 11.31 

4. Shifts and thermodynamic formalism 

When 8 < | the L 2 spectral theory of Lax and Phillips p2] is not 
available and we use symbolic dynamics approach, in particular the 
work of Lalley [16] . In this section we review the key necessary notions 
and results pertaining to shifts of finite type. 

A shift of finite type is defined as follows. Let A be an irreducible, 
aperiodic / x / matrix of zeroes and ones, called the transition matrix. 
Define £ to be the space of all sequences taking values in the alphabet 
{1,2,,...,/} with transitions allowed by A, that is 



E = {x e • • • J} ■ A(x n , x n+1 ) = 1 Vra}. 

71=0 

The space E is compact and metrizable in the product topology. 
Define the forward shift a : E — > E by (ax) n = x n+ i for n > 0. 

Let C(E) be the space of continuous, complex- valued functions on 
E. For / e C(E) and < p < 1 define 

var n / = sup {\f(x) - f(y)\ : Xj = ^ forO < j < n} ; 
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\f\ p = supvar n (/)/p™ 

n>0 

^ p = {/GC(S) : |/| p <oo} 

Elements of T p are called Holder continuous functions. The space 
jF p , when endowed with the norm || • || p = | • \ p + || • is a Banach 
space. 

For /, g G C(£) define the transfer operator Cfg G C(E) by 
^(z) = efiy) 9(y)- 

y:ay=x 

For each p G (0, 1) and / G JF p , £j : T p — > JF p is a continuous linear 
operator; if / is real- valued then £/ is positive. 
Denoting 

S n f = f + f-(T + ...f-(T n -\ 

we have 

(C}g)(x)= eSnfiv) 9(y)- 

y:a n y=x 

The following result is due to Ruelle; a proof can be found in [23] or 

Theorem 4.1. Let f G T p be a real-valued function. 

(1) There is a simple eigenvalue Xf > of Cf : T p — > JF p toif/j 
strictly positive eigenfunction hf. 

(2) 77je rest o/ £/ie spectrum of Cf is contained in {z G C : |z| < 
A/ — £:} /or some e > 0. 

(3) There is a Borel probability measure Vj on £ sttc/i t/iai £}^/ = 

(4) ///i/ zs normalized so that j hfdvj = 1 then for every g G C(S) 

lim \\Xj n C n f g-([ gdu f )h f \\ oo = 

(5) There exist constants C\,E\ such that for all g G T p and for all 
n 

\\Xj n C]g - ( J gdu f )h,\\ p < 0,(1 - etfWgW,. 

The pressure functional is defined by 

P(/) = sup J fdu + H u (a), 
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where the supremum is taken over the set of cr-invariant probability 
measures and H u (a) is the measure theoretic entropy of a with respect 
to v. We have (see [23] or [27]) 

P(/)=logA / . 

A measure fi is called the equilibrium state or the Gibbs measure 
with the potential / if 



J fdn + H li (<r) = P{f). 



For / G T p Gibbs measure ///is the unique a- invariant probability 
measure on £ for which there exist constants < C\ < Gi < oo such 
that 

c < f±f{y eZ:y i = x i ,Q<i<n} 

\] n e W {S n f(x)} ~ 2 - 

As will become clear in the next section, the analyticity properties of 
the map z — > C z f, z G C will play crucial role in the proof. For / G T p 
fixed, real-valued function, such that S m f is strictly positive for some 
m, the quantities C z f, X z f, h z f, v z f will be abbreviated by C z , X z , h z , 



5. Resolvent of transfer operator and lattice count 

PROBLEM 

Let A be a Fuchsian group with no parabolic elements (this condition 
is automatically satisfied in the case 5(A) < | - see [TTJ), generated by 
k elements g ly . . . ,g k C SL 2 (Z). We identify A with £*, defined as the 
set of finite sequences in the alphabet {^i, gj 1 , . . . , g k , g^ 1 } (I = 2k) 
with admissible transitions. According to Series [33] , this may be done 
so as to obtain a shift of finite type. 

Let w G H(= D), and suppose it is not a fixed point for any 7 G A; 
let du denote hyperbolic distance. For x = xi,...,x m G E*(= A) 
define 

(5.1) t(x) = d H (0, x x ... x m w) - d H (0, x 2 . . . x m w). 

The left shift aoiiE corresponds to the Nielsen map (see (33]) F : 
L — > L, where L denotes the limit set of A. 
Recalling that 

(5.2) S n r = T + T-a + --- + T-a n -\ 
we have 

(5.3) S n r(x) = d H {0, x 1 . . . x n x n+l ...w)- d H (0, x n+1 ...w). 
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For a E R, x E E*. 6 : -»• E, let 



oo 



(5-4) iV(a,x)=^ Mv^iSnrMKa}- 

n=0 y:cr n y=x 

Clearly 

(5.5) N(a,x)= J2 

ydh,d}j{0,yxw)— dfj(0,xw)<a 

where in the summation y is restricted so as to make yx admissible. 
In particular, for <p = 1 

N(a, x) = | {7 G A I 7xu>) — cfsr(i, rcu?) < a}|. 

Returning to (15.4p . one has the renewal equation 

(5.6) N(a,x)= N(a-T(x'),x') + 4>(x)l {am 

<r(x')=x 

(cf. PI (2.2)]). 

The link with the transfer operator comes by taking the Laplace 
transform of (15.61) . Defining for ^tz < — C 

/oo 
e az N(a 1 x)da 1 
-00 

equation (15. 6p gives the relation 

(5.8) F(z,x)= e ZT{x ' ) F{z 1 x') + ( ^-. 

cr(x')=x 

Thus we have 

(5.9) (I-C z )F{ Zi x) = ^-. 

z 

This leads us to the study of the resolvent (I — C z )~ l . Before stating 
the results of Lalley [16] and Naud [21] for £ z \f p (T;) (Theorem 15.11 be- 
low) we recall the following reinterpretation of the Hausdorff dimension 
of the limit set in terms of the pressure functional (see [23J or [27J). 
Because r is eventually positive, the variational principle implies (see 
[23] or [27]) that the pressure functional P(—xt) is strictly decreasing 
and has unique positive zero. Define 5 by P(—5t) = 0, that is, 

(5.10) A_i = l. 

Theorem 5.1. (1) There is e > such that for ^tz < — 5+e, z ^ U 
(U a suitable neighborhood of —5 ), we have 

(5.11) ll^|^(Ej||p<|9fz| 2 e- m . 
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(2) For z G U decompose on T p (E) 

(5.12) C z = \ z (v z ®h z ) + C" z , 

(where z — > A 2 , z — > h z , z — > z/ 2 are holomorphic extensions to 
U, satisfying 



£zh z — X z h z , ^z V z — ^Z^Zl 



J h z dv z = 1). 



T/ien 

(5.13) \\(C'T\\ P < e- £n for zeU. 



Part [T] follows from the discussion in Lalley [161 P- 25] (in the case 
when r is non-lattice) and Theorem 2.3 in Naud (21] to provide (15.111) 
when \5sz\ is large. Naud's work build crucially on the approach of 
Dolgopyat [8]. Note that Lalley does not give explicit estimates on 
\\Cg\\ for ^z = —S and ^sz — > oo and certainly no bound of the strength 
of Theorem 2.3 in Naud [2T]. 

Part [2] is Proposition 7.2. in 



6. Lattice count in congruence subgroups for 5 < \ 

In this section we modify the setup discussed in the preceding two 
sections to the setting of congruence subgroups of A — the modification 
is analogous to the one preformed in the case S > ~. 

Fix modulus q such that 7r g (A) = SL 2 (g). Instead of considering 
functions on S (as in Lalley [16] ) we consider functions on S x SL 2 (g). 

For / G C(S x SL 2 (g)) define 



max j E \f{x,g)\' 

,<?eSL 2 (g) 



Var„/ = sup ^ ( ^2 \f(x,g) - f{y,g)\ 2 j \ xj = %-forj < n 

Var n (/) 



I/Ip = SU P 



Let jF p = J- p (E x SL 2 (g)) denote the space of p-Lipschitz continuous 
functions with the norm 
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Let r : — > M be given by ( 15. ip and consider the "congruence 
transfer operator" .M z = -M 2T on x SL 2 (g)): 

(6.1) A*,/(a;, <?) = £ e 2r(i ' x) /(*, ft*?), 

i=i 

where z G C and the summation is restricted so as to make (i, x) 
admissible. 

Thus our M. z differs from the one considered in [16] in that it acts 
on functions on S x SL^g) rather than on functions on S: the reason 
behind this difference is the same as in the proof of the spectral gap 
when 5(A) > ~. 

We have 



M 2 J(x,g) = ^2e^^\MJ)((i 1 ,x),g il g) 



il=l 
I 



£ e z ^ +T ^^f((z 2} t 1 ,x);g l2 g n g) 1 



11,12=1 



and in general for the n-th iterate we have 
(6.2) 



M n J(x; g)= e^'"^^*-- 1 -' 1 '*) 4 -"^^^^ • • • <i, x); g in . . . ^y), 



ii ,...,i„=l 



where again the summation is restricted to admissible words. 
From Ruelle's theorem (Theorem 14.11) it follows that 

l 

3^ 'y * e 3?z(r(i n ,...,ii,x)+T(i n _i,...,ii,a;)H \-r(ii,x)) ^ 

il,...,i n =\ 

for large n. 

Let ip be a function on SL 2 (g). Returning to (15.41) . (15.51) we let 

(6.4) N(a,x) = J2 VMV)) 

y£A,dfi(0,yxw)—dfj(0,xw)<a 

and F(z,x) its Laplace transform defined by (15. 7p . 
Then 

(6.5) F(z, x) = f(z, x, ir q (x)), 
where f(z,x,g) satisfies 

(6.6) (l-A,.)/ = I®£ 
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and A4 Z is the congruence transfer operator introduced above (note 
that obviously 1 <S> p is in J r g (T l x SL 2 (g))). 
Our aim is to evaluate 

(6.7) N(a)= Yl VMV)), 

y£A,dH(0,yw)—dn(0,w)<a 

which gives the sum of p on the mod q reduction of the hyperbolic ball 

{y e A | d H (0, yw) - d H (0, w) < a}. 

Our goal now is to obtain the appropriate extension of Theorem 15.11 
to the setting of congruence subgroup. As is to be expected, it is at 
this point that expansion property will play a crucial role. 

7. Expansion and L 2 flattening 

Let p be a symmetric measure on G = SL 2 (p) (for the sake of expo- 
sition we first consider the simpler case of prime p) and consider the 
convolution map T : L 2 (G) — > L 2 (G), given by p — > pi * p. Decom- 
posing the regular representation of G into irreducible representations 
if follows from the result of Frobenius [H] that each eigenvalue A of the 
convolution restricted to Lq(G) occurs with multiplicity at least 
Trace calculation yields therefore 

IE€G 

Hence 



(7-1) |A| < y — HalGI*. 

Recall also the L 2 - flattening lemma proven in [3]. Let p G V(G) 
satisfy 

(7.2) \Moc<P~ T 
for some r > and also 

(7.3) /i(aGi) < p- T 

for all cosets of proper subgroups G\ of G. Given k > there is 
I = 1(t, x) G Z + such that 

(7.4) ||/i {0 || 2 < |G|- 1/2+K . 

Denote p\x) = p^x' 1 ). Since p* p' also satisfies f)7.2p . (17.31) we have 
by dZ3D that 

(7.5) ||(//*/i)«|| 2 <|Gr 1/2+ * 
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Consider the convolution operator Tip = p! * p * ip and let A be an 
eigenvalue of T on Ll(G). Hence A' is an eigenvalue of T l on Ll(G) 
and applying ( 17. II) with p replaced by (// * implies 

if we take k < \. Consequently 

(7.6) |A|<p _ s. 
This means that if ip G Lq(G), then 

\\H * H * ip\\2 < p 41 || vita 

and hence 

——11 n 

(7.7) \\fJL * ip\\2 <p sl \\ip\\2- 

We proved that if p G P(G) satisfies (17. 2p . (17. 3p for some r > 0, then 

(7.8) II/** vita <P~ r 'lMta, if e L 2 (G) 
for some r' > 0. Therefore if p G A4+((jr) satisfies 
(7-9) IML < P~ r ||/i||i 

and 

(7.10) H(aGi) < P -T ||/*||i 
for cosets of proper subgroups G\ then 

(7.11) ||/** vita <P~ r 'll/*||||vlta, veL 2 (G). 

More generally, let p G -M(G) and decompose p = p + + Estimate 

||/* * vita < ll/*+ * vita + 1|/*- * vita- 
Assume /x satisfies (17.91) and (17.101) . If \\p+\\ > p~^ ||/x||, we have 

||/*+||oo < ||/*IU < P~^||/*+||i and /i + (aGi) < p~5||/i + ||i 
and (17. lip implies that for ip G Lq{G) we have 

||/*+ * vita < ^ r '||/i+||||vlta < P^'lMlllvlta- 
If ||/*+|| < P ^||/*||; then obviously 

||/*+ * vita < || /x+ 1| || vita < P~^||/*||||vlta- 

Hence (1711]) holds. 

The same considerations apply for p G .Mc(G); consequently we 
obtain the following result. 
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Lemma 1. Given k > 0, there is k! > such that if ji G M.c{G) 
satisfies 

(7-12) ||a*IIoo <P' k \H\i 

and 

(7.13) \lA( aG i) <P~1Mli 
for cosets of proper subgroups G\ of G, then 

(7.14) ||/i*^|| 2 < C^ K '||/i||i||^|| 2 for (p E Ll(G). 
Here p is assumed to be sufficiently large. 

We have a similar result for G = SL 2 (g) with q square- free (see [I] and 
[37]). We make the following decomposition of the space L 2 (SL 2 (g)). 
For qi\q, define E qi as the subspace of functions defined mod q± and 
orthogonal to all functions defined mod q 2 for some (^(Zi, <?2 7^ Qi- 
Hence 

(7.15) L 2 (SL 2 (q))=R®@E qi , 

which is, in fact, the generalized Fourier- Walsh decomposition corre- 
sponding to the product representation 

SL 2 (g) = nSL 2 (p). 

p\g 

Let Pi (respectively P qi ) be the projection operator on the constant 
functions (respectively E qi ). 

Lemma 2. Let q be square free and G = SL 2 (g). For \i 6 M. C {G) and 
qi\q define \ \\^ qi {^)\\\oo to be the maximum weight of over cosets of 
subgroups o/SL 2 (g 1 ) that have proper projection in each divisor of q\. 
Given k > 0, there is k' > such that if fi satisfies for all q±\q 

(7-16) WkMWloc < qi K \\v\\i 

then 

(7.17) \\fi*<ph < Cg -re '||//||i|^||2 for ipEE q . 

8. Bounds for congruence transfer operator 

Our goal is to obtain a bound for powers of transfer operator ||A^^|| P 
for the family of congruence subgroups. Recall that M. z acts on func- 
tions on £ x G, so in order to apply Lemma [2] we need to decouple the 
variables. Returning to f)6.2p . fix m < r < n such that m = n—r ~ log q 
to be specified. Write 

M n J(x,g) = 
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(8.1) 
i 

e «(r(W..,*^)+r(*»-i,...,ii^+-+r(ii^) / (( in . . . ); ^ . . . g iy g) 

il,...,tn=l 

(8-2) +0(Xl z \f\ pP r ), 

where the error term refers to the L^ G ^(S)-norm. 

Fix then the matrices i n , . . . , i n - r +i and consider the function ip on 
G defined by 

<f{9) = /((*n> • • • > in-r+i, 0),g in ... g in _ r+1 g). 

We assume f(x, •) G E q for each a;; hence ip G -Eg. 
Our aim is to apply Lemma [2] with 

(8.3) fi= e z{T{in '-' il ' x)+ - +T{il ' x)) 5 9in _ r ... gn . 

Thus by (16.31) we have 

(g 4^ H^ll < \m e ^(T(i„,...,i„_ r+ i,0)+---+r(i n _ I . + i,0)) e , s R2i^ 

where we used the inequality 

(8.5) \T[i n , . . . , z n _ r _|_i, i n — r , . . . ,i\,x) T[i n , . . . , z n — r+i , . . . ) | < ■ 
We bound ||/i||oo, which amounts to estimating 

(q Q) \ e 3tz.(T(i n ,-~,ii,x)+-+T{i 1 ,x)) 

y ' ; (El ^ 

9im-9n=9 

where g is fixed. (We use here the fact that the relation g im . . . g ix = 
gmodq is equivalent to g im . . . g it = g because of the restriction on m). 
Also because of the index restriction on the transition matrix A, the 
condition g im . . . g ix = g specifies (i m , ... G E so that estimating 
(18.61) amounts to bounding 

\ ^Stz{r{i n ,...,ii,x)-\ hr(ii,x)) 

(8.7) <m 

~ J\ _m e ^ 2 ( T ( i mv,H>a;)H br(ii,x)) 

for fixed (i n , . . . i\, x) G E. Thus 

(TO = A '■'M'liA,,. „.,,(,•) < A- m ||A^ 

with (f> any function on E satisfying 4>(i m , . . . , ii, x) = 1 
By Ruelle's Theorem (Thm. 14.11) . 

IIA-" M m </>- {Udv)h\\ p <{\ -e x y 



pi 
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implying 



\\- m M m <t>\u < C (j <j>du + (i - e.ruw^ 



We may now choose suitably, so as to obtain an estimate 

(8.9) A-IA^IL < e" cm . 
Hence, 

(8.10) (EE}, (EI} <q- K . 

More generally, we also need to evaluate 1 1 1 ?r 9l (a*) 1 1 1 oo f° r Qi\q- It 
turns out that the issue reduces to the previous one, using the following 
observation (cf [3]). Let H < G and it p (H) < SL 2 (p) proper for each 
p\qi- Then we can assume the second commutator of 7T P (H) to be trivial 
if p\qi and hence the second commutator of H to be trivial (modgi). 
Take mi < m, mi ~ log q± so as to ensure that words of length 2mi 
have norm less than q±. Using properties of the free group (see [3] ) , it 
follows from the preceding that the number of (i mi , ■ ■ ■ G S such 
that gi m . . .g% x G aH is bounded by 0(mf ) for some constant C. Hence 
we may invoke the estimate on (18. 7ft with m replaced by vii\ to obtain 
also 

|||7r ?1 ( /U )||| 00 <gr K -(lH3D. 

Applying Lemma [2J it follows that 

||/i*</?|| 2 < <r K '\\M\<Ph < g- K 'A^ 2 e^ (r(i "'---' i "- r+1 ' 0)+ --- +r(i "-'- +1 ' 0)) e^^ 
or 

(8.11) 

q~ K ' X m e^ z ( T ( in '''' ,in ~ r+1 ' ^ + ''' +T ( in - r+1 ' ^e^ z ^ \\f\\oo 
Summing (18. lip over i n , . . . , i n - r+ i implies by (16.31) again 

(8.12) 
i 

|| e*(r(i»,...,i^)+r«n-l,..,i^)+-K(i^)) /((in Q) . ^ ^ 

ii,...,i n =l 
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Therefore it follows that if n > log q 

, \\MV\\ L - w<X&(q-«\\f\\ M + fr\f\p) 

8.13 l(G) 

<^- K '(\\f Woo +P^\f\ P ) 

for 

(8.14) / E T' p = J-'pd Ce q (E). 

Note that in (18.131) there is no restriction on Qz. 
We also need to estimate \M.™f\ p - 

Let x, y € E be such that x« = for < z < Z. Estimate 
|A<:/(a:^)-A<J/(y^)|< 

(8.15) 

E e^^-^^ + - + ^^)|/(i n , . . . , g in . . . g h g) - f(i n , • ■ • , hy\ n,, ■ ■ ■ //,//) I 
(8.16) 

+ | ^ (^..MH-^W) _ e.(r(W.,ii,»H-+r(«i.»))) /(<„,..., i x y; (&,... 

ll,...,ln 

Clearly for the first term we have 

(8.i7) ^m<\i z \f\ pP n+i . 

To estimate ( 18.161) we repeat the argument leading to ( 18.131) . Thus 
we bound ( 18. 16ft as follows 
(8.18) 

ii,...,i n 

(8.19) 

+ P r |/lp I ^ e 2 ( T ( i "v,n,a;)+---+-r(n,x)) _ e «(r(i n ,...,ti,y)+-+T(ii,j/))^ |_ 

il,...,i n 

Estimate 
(8.20) 

( e z(r(i n ,...,i 1 ,x)-\ hr(ii,£c)) _ e «(r(i n ,...,ii,j/)H Hin^U I 



e 3?z(r(i n ,...,n,x')+T(ii,x))|^ _ g 2 

'1 ) ■ ••^71 

< A^(l + |^|)|r|p(p^ + ■ • ■ + P 1+i ) < A^^^lrl^; 



ll,.--, V. 

< \~"* o^M 1 "^ ' I ) + ~l ' I • I ) I -| _ , :lrl',. 'i-'/H r r( M .//)- r( /'„ ,,..#•) ri/j.../-.!) 

ll,...,l n 

1 + IQfzl 
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therefore 

(8.21) (EHD < X n ^^\r\ p p l+r \f\ p . 

To bound (I8.18P we apply again the convolution estimate on G from 
section [71 Consider the measure 
(8.22) 

y — ^ e z(r(i n ,...,i 1 ,x) + ---+r(i 1 ,x)) _ e z(r{i n ,...,i 1 ,y) + -+T(i 1 ,y))^ ^ ^ 

with i n , . . . i n -r+i fixed. 

Repeating fl8.20p gives (with m = n — r) 

J 

(8.23) \\v\\ < A m (l + |^|)| r | p _i_ e ^M^.-..,in-r + i,o)+---+r(i„_ r+ i,o))_ 

Also, as above, we have 
(8.24) 

^ = \ z(T(i n ,... ,H,a:)H hr(ii,x)) _ z(r(i„,...,ii,j/)H hr(ii,j/))| 

fl8T23|) " l|o ° fl8T23|) ' 1 

< ^-m e 5Rz(r(i„,...,ii,x)H h-r(«i,x)) 

and 

(8 ' 25) ^^lll^i^llloo <C K for gi |g. 

Therefore, by the results from section [7J we obtain (with if defined 
as in section [7J that 

\W*v\\m G )<q- K 'mB\\f\\oo< 

(S 26) ' 

^-K^ym^ | |Cfo|)| r | p ^ c Kz(r(i n ,...,ii n - r+ i,0)H hr(i n _ r+ i,0)) 

Summation over i n , . . . , z n _ r+ i gives then 



(8.27) ||(EIHD||p (G) < q- K 'X n (l + |^i)|rU||/|U. 

From (JEHD, (lOTT) . (K2T\\ . it follows that 
(8.28) 

\\M n J(x, -)-M:f(y, .)||p (0) < p'A^ (p»|/| p + + |3z|)|.f |, + g" K '(l + 



Therefore, if n > log q we have 
(8.29) \M»f\ p < CXl z q- K ' (H/IU + p n ' 2 \f\ p ) (1 + 
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Take n such that 

(8.30) n~logg + Clog(l + |Sz|) 

for a suitable constant C. It follows from (18.131) . ( 18. 29ft that 

(8.31) \\M n J\U + p*\M?f\ p < Xl z q- K '(\\f\\oo + P%\f\ P ). 
Iterating (18.311) shows that if / G J 7 ', then for all m G Z + 

(8.32) \\MT f Woo + P^\MTf\ P < >^q- mK '\\I\U 
and hence 

(8.33) \\MTf\\ P < >^q- mK 'q{l + MII/IU 
where n is given by (I8.30p . Thus for m > 1 

(8-34) ||^kllp<A^g-^'g(l + l^l)- 

We distinguish two cases: log(l + < logg and logg <C log(l + 

The conclusion is the following: 

Lemma 3. Notation being as above, there is e > such that 

(8.35) II-M^Mp < <7 C e" £m A^ |9fe| < g 
and 

(8.36) ||AC|^|| P < |^| C e" e ^^ m A^ if \%z\ > q. 

9. Resolvent of congruence transfer operator 

We now use Lemma [3] to estimate the resolvent (/ — M. z )~ x on J 7 '. 
By ( 16.51) . (16.61) this will provide us bounds on F(z, x), assuming G E q . 
Take < — 5 + E\ such that 

(9.1) A_ 5+£l < ei 
with e > from (18.351) . If jQ^I < g, we obtain 

(9.2) W-M z y\ lp \\ < q C J2 e ~ £mX ^ % 
If > q, we impose the restriction 

logg 



(9.3) ®z<-5 + e 2 - 

log \ isz\ 

with e 2 > small enough to ensure that 

_ e log? _£ log q 

\<ft z e lo si az i < e 2 lo si az i . 
Under this restriction on z, we obtain from (I8.36P that 

(9.4) ii(/-A<.r 1 kii<i^i - 



(9.5) < -5 + e 2 min 1 
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In summary, we proved the following 

Theorem 9.1. The resolvent (I — M. z )^ l \r' p is holomorphic on the 
complex region D(q) given by 

logd^zl + 1 

(with 62 independent of q) and satisfies the estimate 
(9.6) \\{I-M z )- l \^<{q-r\^z\)°. 

Returning to (16.51) . (16. 6p . it follows that for <p e E q the Laplace 
transform F(z, x) of N(a, x) is bounded by 



(9.7) l^^)l< W U 1Mb 





\$Sz\) c 




z\ 





for z satisfying (19. 5p . 

To extract information about N(a) we apply Fourier inversion to 
(15.71) . following the argument in [TBI P- 31] (but with a different class 
of functions k). 

Specify some smooth and compactly supported bump function k on 
R. We get from (ETTJ) 

(9.8) / k(t)e- 5t N(a + t)dt = e Sa f e- iad k(-i6)F(-5 + i6)d6, 

J —CO J 

where 

k(z) = J e zt k(t)dt 

is an entire function. 

Note that \k(i6)\ is rapidly decaying since k is smooth. 

In fact, proceeding more precisely, fix a small parameter 7 > (the 
localization of k) and consider functions 

(9.9) k 7 (t) = he , 
where K is a fixed smooth bump function such that 

(9.10) (K = l, 



(9.11) supple [-i i], 

(9.12) \K{\)\ < e- |A|i for |A|->oo. 
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Hence 

(9.13) \k(z)\ < e~ hlz ^ for \toz\ < O(l). 

Returning to (I9.8p . modify the line of integration = to the curve 

z{6) = w(9)+i9, 

where 

(9.14) w(r) = le 2 mm(l, l ° g<1 



2 Z V 'log(l + |0|), 

so as to remain in the analyticity region given by Theorem 19 . 11 
We obtain 



e 5a / e -°*V)k( z (e))F(-5+z(0))d6 = e 5a / e- awie) - iae k(w(9)+i6)F{-5+w{6)+i 



CO 



which is bounded by 

(9.15) |M| 2 e 5a / e - aw{e) e~ { ^(q+ \6\fd6, 

CO 



CO 



applying fIDTTBl and (RT7|) . 

From the definition of u>(#) it is clear that 

(9.16) Q32D, (EE]) < e 5a gV C exp ( -a£ 3 min ( 1, I j |M| 2 . 



log^ 



This proves (replacing k{t) by e 5t k(t)) 



Proposition 1. Let tp G E q and N(a) given by (\6.7\i . Then 
(9.17) 

7 /2 



k 1 (t)N{a + t)dt 

- 7 /2 



< g c 7 ~ C exp ( -ae 3 min ( 1, ] ] e 5a |M| 2 . 



10. Bound for the error term 

Next consider the case where in (15.41) . 0=1 (the constant function). 

Here we consider simply the action of C z on ^>(S) exactly as in [16], 
but we use the stronger estimates on (J — Cz)" 1 following from (15. lip , 
given by [2T] . 

If ^tz < —5 + £4 (with 64 small enough) and z ^ U (some complex 
neighborhood of —5), (15.111) implies that 

(10.1) \\(i-c z y l \\<\zz\ 2 . 

For s EU, apply (15TT2|) . Thus 

r z = \ n Av z ® h z ) + (c'T, 
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where ||(£") n |l < e £n by (1Q51) . 
Hence for z £ U 

(10.2) (/ - c z y l = -^(^ ® h z ) + (/ - c" z y l 

with (J — -C") 1 holomorphic (this is Proposition 7.2 in [To]). 
Combining (flO~Til . (flTOll we get 

Proposition 2. Consider C z acting onjF p (S). Then for ^lz < —5 + 6$ 

(10.3) (/-/y-i-.—L.^®/^) 

1 — A-, 



is holomorphic and bounded by Cd^zl 2 + 1). 

Let n z = v z ® h z . The function has a pole at z = —5 with 

residue 

1 _ 1 
(£K)\z=-6 Jrdfx_ s ' 

Consequently 

_ x Z/_ 5 <g> 1 



(10-4) (I-C ZJ , 

J rdfi-s z + o 

is analytic for $lz < —5 + £5. 
Letting 

(10.5) N(a)= Yl 1 

s/eA 

dH{0,yw)—d[i (0,u>)<a 

and 

(10.6) F(z) = J e az N(a)da, 
it follows from O , fliTTD that 



(10.7) F(z) = -(I - C z )-H = h -f~^ 1 + 

z J rdfi-s z + 

where G(z) is analytic on < — 5 + e 5 and bounded by CQ^^I 2 + 1). 
As in section [9] we have 



J k J (t)e- 5t N(a + t)dt = e 5a J k^{i6)F{-5 + i9)e~ iad d6 
= e 5a (c J e~ iaf) ^{16)^6 + J e-^k^Gi-d + lO^j 



(10.8) 
where 

(10.9) C = = W) 



J rdv-t 
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The second term in (110.81) is estimated by moving the line of inte- 
gration = to 9ftz = |e 5 . We obtain by (19.131) and the assumption 
on G: 



10.10) 

| J e~ ia9 k^te)G(-5 + i9)\< e'^ a J(l + 6 2 ) e -^ W)h d6 < c 7 " 



3 -55-1 



Also 



f e-^k^i&^de = [ ky{t + a)dt ™ 1. 
Jpv ™ Jo 

Therefore we obtain 

Proposition 3. Let N(a) be given by (|10.5|) . Then 

(10.11) J k 7 (t)N(a + t)dt = C e 5a + o ( T - 3 e^- E6)o ) 

for some Eq > 0. Here Co is a fixed constant. 

11. Proof of Theorem 11.41 

Let ip be a function on SL^g) and let N(a,x) denote the counting 
function given as above by 

N{a,x)= <PMv))- 

y£A,d,H{0,yxw)—dH{0,xw)<a 

What we proved in Theorem 19.11 is that for <p e E q the Laplace trans- 
form of N(a,x) in a (given by (15. TP ) is holomorphic on D(q) given 
by 

(u.i) ^) = {^^<-^^^( s og(l '° g ; +1) )} 

with e 2 independent of q. Let us denote by C z {q) the dynamical trans- 
fer operator on the congruence subgroup A(q). Thus det(l — C z (q)) 
is the dynamical (Ruelle's) zeta-function associated with the congru- 
ence subgroup A(g). Using (I5.9P we have that the Laplace transform 
of N(a,x) is also obtained as the inverse of (1 — C x (q)). Now consid- 
ering the action of C z (q) on jF p (S(g)), recalling the decomposition of 
L 2 (SL 2 (g)) given by (|7.15|) . and applying theorem 19. II to E qi for all qi\q, 
and Proposition [2] to the constant function, we obtain that 1 — C z (q) 
has holomorphic inverse (apart from z = —5) on 

D = D(l)n fl £>(<&), 

91 1? 

where 

D(l) = {z :^z< -5 + e 5 } 



GENERALIZATION OF SELBERG'S THEOREM AND SIEVE 



2!) 



by Proposition [2j Consequently D is given by 
(11.2) D = \z : Uz < -5 + £ 6 min (1, 



log(|3*| + 1) 



1 



) 



for some e$ independent of q, implying that dynamical zeta function 
det(l — C z {q)) has no zeros on D (apart from simple zero at —8). 

Theorem 11.41 now follows from the equality of the dynamical zeta- 
function and Selberg zeta function (Theorem 15.8 in [2]), and the corre- 
spondence between the zeros of Selberg's zeta function and resonances 
(see [26] and Chapter 10 in [2]). 



Propositions [2] and [3] are our basic estimates used in the proof of 
Theorem 11.51 Note that what comes out Proposition [1] will only play 
the role of error terms. 

Fix a modulus q, (q, go) = 1 (with q given by the strong approxima- 
tion property) and q square-free. 

For some element £ G SL 2 (g) we need to evaluate 



(12.1) N(a-q,0 = \{yeA;n q (y) = £ and d H (0,yw) < a}\ 



(replace a by a + d H (0, w) in (16.71) ). 
Recall the decomposition of the space L 2 (SL 2 (g)) in (|7.15p . Writing 



12. Proof of Theorem 11.51 



(12.2) 




we get 



(12.3) 




1 



d[f(0,yw)<a 



(12.4) 





qi\q yeA 

517^1 d H (0,yw)<a 



with 



= Pqi0-g=t) £ E qi- 



Thus 



(12.5) 



SL 2 (gi)|2 
|SL 2 (g)| 
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We use Proposition [3] to evaluate the right-hand side of ( 112. 3ft and 
Proposition Q] to bound terms (j!2.4p . Hence, fixing some 7 > 



;i2.6) 



k 7 (t)N(a + t;q,£)dt 



( ' e Sa + o( 7 - 3 e- ca e 5a ) 



'12.7) +7"° ^ 1i ex P -ca min 1 
qi\q 

91^1 

We estimate (112. 7p as 



SL 2 (g)| 

log<?i\\ |SL 2 (?i)|3 



logSM |SL 2 (g)| 



e 5a . 



(12.8) 

(12.9) 
and 

(12.10) 

assuming 
(12.11) 



^f~ c \a\ c 



e ^ 



( 

£ 

Qi\q 



+1 -C q C e (S-c)a 



,,5a 



E 



<?il<? 

91^1 



* <n(i 

p\q 



< 



exp [J2 



log : 



1 



e log ~ 



- 1 < e 



- 1 



log - -C a. 

7 

Therefore we proved the following result, of which Theorem 11.51 is an 
immediate consequence. 

Proposition 4. Notation being as above, we have 
(12.12) 

k,(t)N(a+t;q,Odt = — — - {C x + o(j- C e ^ )j +1 ~ c q c e^ a , 



where we assume 
(12.13) 



1 a 
log - < 



7 log a 

We remark that what is really required for sieving applications is a 



bound for the ratio 
(12.14) 



/ ky(t)N(a + t',q)dt 
J ky(t)N(a + t)dt 
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of the form 

' + 0(e~ £a q c 

or 



|SL 2 (g)| 

(l + 0(e~ c ^)) +0{e- £a q c ^ 



|SL 2 (g)| 

To bound the ratio (112.141) it suffices to use Proposition CD (which builds 
crucially on the generalized expansion result given by Lemma W) com- 
bined with the result of Lalley [IE]. Of course, the results of Dolgopyat 
[8] and Naud [21], [22] are necessary to establish Theorem 11.41 which 
is of independent interest. 

13. Proof of Theorem 11.61 

13.1. Combinatorial sieve. As in |J], we will make use of the sim- 
plest combinatorial sieve which is turn is based on the Fundamental 
Lemma in the theory of elementary sieve, see [14] and [13]. Our for- 
mulation is tailored for the applications below. 

Let A denote a finite sequence a n , n > 1 of nonnegative numbers. 
Denote by X the sum 

(13.1) ^a„ = X 

n 

X will be large, in fact tending to infinity. For a fixed finite set of 
primes B let z be a large parameter (in our applications z will be a 
small power of X and B will usually be empty). Let 

(13.2) P = p=Y[p 

vis 

Under suitable assumptions about sums of A over n's in progressions 
with moderate-size moduli d, the sieve gives upper and lower estimates 
which are of the same order of magnitude for sums of A over the n's 
which remain after sifting out numbers with prime factors in P. 
More precisely, let 

(13.3) S(A,P):= a »- 

(n,P)=l 

The assumptions on sums in progressions are as follows: 

{Aq) For d square-free, and having no prime factors in B (d < X), 
we assume that the sums over multiples of d take the form 

(13.4) a n = P(d)X + r(A,d), 

n=0(d) 
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where (3(d) is a multiplicative function of d and 

forp ^ B, (3(d) < 1 — — for a fixed c 1 . 

Ci 

The understanding being that (3(d)X is the main term and that the 
remainder r(A,d) is smaller, at least on average (see the next axiom). 

(Ai) A has level distribution D = D(X), (D < X) that is 

S |r(d,j4)l<< fi^ for all3>0. 
(A 2 ) A has sieve dimension t > 0, that is for a fixed c 2 we have 



X] Z^(p) logp - Uog 



M)<P<Z 



<C 2 



for 2 < w < 2. 



In terms of these conditions (A ), (A ± ), (A 2 ) the elementary combi- 
natorial sieve yields: 

Theorem 13.1. Assume (A ), (A\) and (A 2 ) for s > 9t and z = D l l s 
and X large we have 



(13.5) 



X X 
— < S(A, P z ) < — . 

(\ogxy (\ogxy 



13.2. Applying the sieve. Now let A be a Zariski dense subgroup of 
SL(2, Z) and let / G Zfxjj] be weakly primitive with t(f) irreducible 
factors. The key nonnegative sequence a n to which we apply the com- 
binatorial sieve is defined as follows: for n > we let 



;is.6) 



a n (T)= L 



7SA 
|7|<T 
l/(7)l=n 



The sums on progressions are then, for d > 1 square free 
(13.7) £ a n (T) = 1 = E E L 

n=0(d) 



7 GA;| 7 |<r 

m=o(d) 



peA/A(d) 7 eA(cZ) 
/(p)sO(d) | 7 |<T 



GENERALIZATION OF SELBERG'S THEOREM AND SIEVE 33 

Consider the case of 5 < 1/2; the case of 5 > 1/2 is similar and simpler. 
According to Theorem 11.51 we then obtain 

(13.8) 

^a n (T)= u"j {l + 0\T-^^jj +0(d c T 25 - £l ) 

n=0(d) peA/A(d) ' d ' 

f(p)=0(d) 

> 

= x^j + O j + O (JA^X 1 ^ 

where 

(13.9) X = J2*k(T)= J2 !» 

fcGN 7€A 
|7|<T 

Arf is the reduction of A mod d, and A^ is the subset of A^ at which 
f(x) = mode?. 

Using strong approximation theorem [19] and Goursat lemma as in 
[4] we obtain that outside of finite set of primes S(A) we have A p = 
SL 2 (F p ) and A — > A dl x A d2 is surjective for (di, d 2 ) = 1 and d\d 2 square 
free. Let 

(13.10) (3(d) = M 

\ A d\ 

Using Lang- Weil theorem [TS] as in [I], we obtain 

(13.11) |Ai|«rf 2 
and 

(13.12) + 

Hence we have 

(13.13) a «( T ) = l 3 ( d ) X + r ( A ' d )' 

n=0(d) 

with 

(13.14) r(A, d) < ix 1_ ^iogio g x + d c+2 X 1_ &. 

Verification of (Aq) is completely analogous to Proposition 3.1 in [I]. 
Regarding the level distribution (Ai) we have that 

(13.15) V |r(A, d)| < X 1 - w^ix + £,0+3^-1-^ ^ 

^ (logX) B 



d<£> 
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for any B > as long as 

25 

(13.16) D <X T with t < 



(C + 3)ei 

Finally, to verify the third axiom concerning the sieve dimension, we 
have, using (113.121) . that 
(13.17) 

£, 8 ( P ).o gP ^ £ (*^ + o te)) =tl0g £ + o(1) , 

w<p<z w<p<z ^ \ / / 

which establishes (A2) with the sieve dimension being t. 
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